Cross-Population Comparability: an introduction

The realization that numerous factors contribute to how individuals make judgements and respond within interview based surveys is not new (Jobe & Mingay 1989; Kahneman, Slovic & Tversky 1982). Over the past 50 years or so, investigations in the area of " respondent bias " have documented various sources of error, i.e., factors that may contribute to the difference between what is truth, what is perceived or judged and what is ultimately reported or observed. Earlier work primarily focused on describing these different factors (see Streiner & Norman 1995), while latter work investigated the cognitive processes that contribute to such biases (see Kahneman, Slovic & Tversky 1982). Some of these factors and processes include: Social desirability or secondary gains (purposively under-or over-reporting level of health status) Aversion to end points or central tendency bias (e.g., reluctance to use the extreme categories on a scale, such as very bad and excellent) Halo effect (e.g., the general impression influences how specific traits or states are assessed, and may reflect that raters are unable to evaluate more than a few dimensions distinctly from a general dimension) Positive bias (e.g., most likely to agree or acquiesce with interviewer) Framing effects (e.g., choice between alternatives depends on how the question is framed rather than the actual content of the question, and the response therefore measures other attributes of the individual such as risk aversion or risk taking, rather than those intended by the original question) Judgmental heuristics that are biased (e.g., common strategies people use to make judgements that are influenced by other factors that serve to decrease the accuracy of responses) Combining perspectives from a variety of fields, including psychology, decision sciences and statistics, a classic article by Tversky & Kahneman (1974) discussed that people rely on a limited number of heuristic principles in order to simplify complex decision making. Three common heuristic principles include representativeness (e.g., if asked about A, a person uses information on B, and bases the decision making process concerning A on the degree to which A is representative or resembles B), availability (e.g., a person assesses a situation based on the instances or occurrences that can be brought to mind based on one's own experiences), and adjustment and anchoring, (e.g., a person estimates an initial response or anchor starting value, and then during the decision making process makes adjustments – usually partial or incomplete – to that starting value …